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ABSTRACT : 

Data bases for question answering systems are models of reality. 
Such models can be considered to be sets of assertions. Sophisticated 
models must. cope with consistency problems due to the Frame Problem. 
Processes that derive answers to questions should be included in the 
model, and question answering systems should be able to discuss such 
processes. Uniform processes are too rigid to support intelligent 
question answering systems. Further understanding of nonuniform 
processes may derive from development of nondeterministic programming 
languages. The SIR model, the CONVERSE model, and the DEACON model are 
compared. Directions for research are proposed. 

This task was supported by: Navy Electronics Laboratory 

Contract No. WR2-9100 
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INTRODUCTION 



The computer offers the possibility of automating information 
and fact retrieval systems* Eventually we will be able to speak 
directly to computer systems in natural language, explaining what 
we require of them much as we would outline a task for a human 
assistant, and engaging in dialogues with them to assure that they 
(or we) understand whatever is necessary. Current technology 
supports the implementation of only limited applications of question 
answering systems; the query languages, subject matter, means of 
communication, and amount of data are all stringently limited. How- 
ever, research on question answering systems and problem solving 
systems has led to some understanding of the linguistic and cognitive 
processes that must be developed in order to expand the capabilities 
of fact retrival systems beyond the limits of current technology. 

Current question answering systems are able to accept queries 
stated in more or less natural English, and answer them by referring 
to a data base. The type of information recovered by such systems 
could also be obtained by performing various counting and sorting 
manipulations on data in the form of tables and lists. Causal, 
qualitative, judgmental, and, generally, deductive information simply 
cannot be provided by these systems. 

Systems designed to deal with specific, limited domains of 
information can be built with considerable knowledge of the domain 
incorporated in the design. These special-purpose systems sometimes 
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provide some deductive abilities, and may rival or exceed human 
performance in deduction and problem solving within their restricted 
areas. However, such general question answering systems as are 
available possess very limited capabilities. In addition, natural 
language processors typically consume large amounts of time and memory 
and can operate only with relatively small data bases. The problem 
of coping with large and diversified data bases is receiving some 
attention, but cannot be considered solved. 

With the development of improved natural language technology and 
related technologies, such as computer speech recognition and 
production, and computer problem solving, it will eventually become 
possible to automate most information and fact retrieval systems; 
but when these developments will occur, and indeed, precisely what 
developments are necessary, are not completely apparent at this time. 

This paper presents an examination of some of the alternatives 
available for the representation and manipulation of data in question 
answering systems, together with a discussion of limitations imposed 
by uniform processes, and compares three specific types of data 
representation taken from three contemporary question answering 
systems. 
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MODELS 



A question answering system must answer factual questions. To 
obtain the necessary facts, the system must have access to a supply 
of information; such a store of information is a model of reality. 

In an applied system, factual information about the world would be 
given to the system to be used in answering questions. 

Figure 1 shows some possibilities for structuring a question 
answering system ^s model of reality. The model reflects only the 
relevant aspects of reality - certainly not all of reality. In 
Figure 1, the first major distinction occurs between direct, mechanical 
models and formal models. Conceptually, a direct model operates by 
mechanically reproducing the relevant features of reality. For example, 
consider the task of proving theorems in elementary geometry. One way 
of recognizing reasonable hypotheses is to draw diagrams that represent 
the hypothesis under consideration. Examination of such drawings often 
reveals the absurdity of an incorrect hypothesis, which can then be 
dropped; effort that might have been spent attempting to prove the 
hypothesis correct can thus be saved. 

Little is known about direct models, and no question answering 
system uses such a model. An interesting research project might 
attempt to develop a direct model for some reasonable subject and to 
use the model as the basis of a question answering system. 

The alternative to a direct model is a formal model. A formal 
model is one in which the relevant aspects of reality are represented 
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Reality 




objects 



Figure 1. Some alternatives in the construction of models of reality 
for use by question answering systems. 
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as assertions in some suitable language. Directed graphs, tables, 

lists, and various logical calculi are examples of such languages. 

The prime advantage of formal models is that the related languages 

are easy to represent in a computer and are relatively easy to 
•k 

manipulate. 

The main disadvantage of formal models is that logically or 
mechanically necessary properties of reality are not logically or 
mechanically necessary in formal models, and must be included as 
additional information. Thus, for example, if you attempt to draw 
a triangle whose interior angles sum to less than 180 degrees, you 
will find that you must bend one of the supposedly straight sides, 
whereas if you simply assert in a suitable language that a particular 
triangle has less than 180 degrees the error may never be discovered. 
This issue is related to the frame problem, discussed in the 
f o llowing sect ion . 

Since current question answering systems use only formal models, 
the remainder of this discussion will be restricted to formal models. 
A formal model consists of a data structure, or language, a set of 
processes for manipulating the data, and a set of conventions for 
interpreting the meaning of the data. For example, the data might 
consist of atomic symbols and lists containing pairs of elements, the 



The representation of very large data bases in formal models 
is not well in hand, but would presumably be even more difficult 
using a direct model, since direct models are not understood at all. 
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convention being that the list consist of attribute-value pairs, and 
the processes being suitable functions for building, changing, and 
reading such data structures. More abstract conventions might be that 
the atomic S 3 mibols represent atomic quantities such as numbers, names, 
and fundamental relations, and that objects and sets are represented 
in the data as description lists. 

Data structures may be divided into the following categories: 
atomic aggregates, formulas, processes, and syntax. (See Figure 1.) 
Atomic aggregates consist of atomic symbols and structures composed 
of atomic symbols. For example, numbers, character strings, arrays, 
and lists can be considered atomic aggregates. Formulas are typically 
in some logical calculus such as a form of the predicate calculus. 
Processes can be used as data objects in some languages, such as LISP 
and PLANNER. For example, Winograd (1971) uses PLANNER to represent 
his data as processes. Thompson’s REL system (Dostert, 1971) stores 
considerable information in an expandable syntax. This feature of 
REL will be discussed presently. 

There is an important distinction to be made within the category 
of atomic aggregates. Typically the atomic symbols represent basic 
quantities, such as numbers and basic relations (e.g., set membership, 
equality). However, the higher level structures may represent facts, 
objects, or combination of the two. For example, the fact that John 
is six feet tall could be represented as (HEIGHT JOHN 6*), and the 
object John, whose wife is Mary, could be represented using description 
lists as: 

2254 (NAME JOHN WIFE 2642) 

2642 (NAME MARY HUSBAND 2254). 
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Clearly, if the data elements are facts, then the real world objects 
are represented indirectly via interconnections between facts, and 
conversely. Ring structures allow both types of connections to be 
made explicitly in the same data structure, at the cost of considerably 
increased linkage. We shall presently see examples of all three 
types of atomic aggregate representations. 

Most current question answering systems are based on data 
structures primarily consisting of atomic aggregates. These structures 
are less expressive and flexible than formulas, for example, but by 
virtue of these very qualities, they are easier to manage. We will 
address two of the factors that limit the development of more 
sophisticated question answering capability - the frame problem and 
the lack of nonuniform processes. The frame problem is more critical 
in highly sophisticated data representations and where changing 
situations are represented. The liabilities of uniform processes are 
sometimes subtle, but pervade current systems. 

The Frame Problem 

The frame problem consists of keeping track of the effects of 
actions. The word ”frame” refers to a frame of reference, so 
that the frame problem is effectively the problem of appropriately 
modifying a frame of reference. The frame problem originates in the 
study of computer problem solving where the machine’s task is to 
construct a sequence of operations that transforms a given initial 
situation into some desired situation. For example, the reknowned 
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Missionaries and Cannibals problem consists of an initial situation 
in which three missionaries, three cannibals and a two-man boat are on 
the left side of a river. The desired situation is that all be on the 
right side of the river, and the legal transformations include all the 
obvious possible actions, except that at no time may any missionaries 
be left outnumbered by cannibals. 

While the frame problem may appear simple, it is extremely 
refractory. If the frame of reference represents a physical situation, 
and the modification is due to some physical action having occurred, 
then it is quite difficult to know just how to modify the frame of 
reference. For instance, if I have just constructed a color television 
from a kit, and I plug it in and turn on the switch, how do I know 
what will happen? On a simpler level, if I am working on the 
Missionary problem and I transport a missionary, a cannibal, and the 
boat across the river, how do I know where the remaining missionaries 
are? While the latter problem is easy, the former is not; further, 
if the former is not difficult enough, consider attempting to 
determine how well the contry’s missile defenses would work without 
actually using them in combat. 

Approaches to the Frame Problem . There are several methods 
proposed for handling the frame problem; see Raphael (1971) or 
Hayes (1971). The most direct is simply to state physical actions 
as mappings from complete state descriptions to complete state 
descriptions. Thus, in the Missionary problem, the action of moving 
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the two people across the river would contain also the action of 
leaving the others where they are. This approach is unworkable if there 
are many actions or many elements in the complete state description. 

Most of the approaches discussed in the literature seem to be 
directed at solving the problem completely, in the sense that the 
updating of the frame of reference be done without error and without 
omission, as much as possible. Hayes (1971), for instance, presents 
a modification of the predicate calculus intended to allow updating 
of frames of reference represented as sets of predicate calculus 
assertions . 

Our view is that: 1) a complete solution to the frame problem is 

too much to expect. Humans do not have anything like a complete 
solution for the frame problem, since they are routinely surprised or 
disappointed at the consequences of their actions, and cannot plan far 
ahead with much detail and reliability. 2) Much expertise about the 
world is contained in assertions about the consequences of actions, 
so that it is appropriate to represent the calculation of consequences 
of actions in the same form as the actions themselves are represented. 
Consequently, a more pragmatic approach seems appropriate. For 
example, the machine could alternate between making actions (in which 
the direct consequences of the actions are recorded in the frame of 
reference), and updating the frame of reference. In the update phase, 
the indirect consequences of the action taken could be computed and 
recorded . 



- 9 - 



The importance of the frame problem will increase as question 
answering systems are expanded to provide deductive and problem 
solving capabilities and to cope with data involving changing facts. 

Question Answering Processes 

In most question answering systems, the question answering 
processes are constructed as integral parts of the system, sometimes 
being fully specified early in the design of the system. Consequently, 
these processes are not within the system’s model. A further practical 
consequence is that the processes are not clearly and distinctly 
defined; the major processes make sense only within the total system 
design, and cannot be understood or discussed effectively out of that 
context. 

If at least some of the question answering processes could be 
shifted into the model, it might be possible for the system to 
accept changes and improvements as additional data. Those processes 
that are not in the model can be changed only by reprogramming. The 
possibility that a system could participate in generating improvements 
by a process of verbal reasoning suggests interesting directions for 
research. 

Processes are programs. Programs may be instruction oriented, as 
most computer languages are. Instruction oriented programs specify 
sequences of elementary actions, and are typically difficult to treat 
as data objects; early list processors such as IPL-V had as one of 
their advertised advantages the fact that programs could be treated 
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as data. Alternatively, programs may specify definitions. In this 
case, program execution is accomplished by evaluating definitions. 

LISP is the prime example of such a language. Processes stated 
def initionally are less distinct from data than are processes stated 
instructionally . 

Both instructional and definitional languages present difficulties 
to the programmer: in typical languages the programmer must specify 

minutely the procedure to be followed. By comparison, in giving 
instructions to a human assistant, a person concentrates on the 
important or tricky steps, and fills in other details as needed. 
Usually a human assistant requires far fewer details than a computer. 

Higher level programming languages allow the programmer to avoid 
specifying as much detail as lower level languages require; indeed, 
this is one of their major advantages. However, the more abstract 
the language is from the details of the computation, the more 
decisions the language processor is required to make. Language 
processors approach each program in the same way, regardless of 
whether the program is large or small, sophisticated or elementary, 
or how long it will run. The language processor applies Just the 
same procedure in each case. Such a processor is referred to as 
uniform . Current language processors and most theorem proving 
programs are uniform processes. 
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The Liabilities of Uniform Processes 



For an example of the drawbacks of uniform processes, consider 
the factorial function and the Fibonacci sequence. Suitable 
recursive definitions of the factorial function and a function FIB(N) 
which computes the nth Fibonacci number are given in Figure 2. If 
a typical LISP interpreter were given these definitions, it would 
require N + 1 applications of the definition to compute N factorial. 
However, since each application of the definition of FIB in which N 
is greater than 1 requires two additional applications of the 
definition, the calculation of FIB(N) generates a binary tree whose 
minimum depth is 1 + N/2 and whose maximum depth is N. Each node 
represents an application of the definition; since there are more 
than 2^^ nodes in this tree, the time required to calculate 

FIB(N) is at least proportional to 2^^ For example, if 

each application of the definition took 25 micro seconds, the time 
required to calculate FIB (64) would be greater than 25*10 ^ sec. * 

2(1 + 64/2)^ = 2^^, which is approximately 8 billion, 

so the time estimate is roughly 25*10 ^*8*10^ sec., or 200,000 
seconds. While no one would attempt to calculate N factorial 
faster than the LISP interpreter, any college calculus student should 
be able to calculate the 64th Fibonacci number in well under 200,000 
seconds. Of course, a simple modification in the program or the 
interpreter would render the FIB function as fast as the factorial 
function, but that is just the point: the rigidity of a uniform 
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Figure 2 



FACT(N) = if N = 0 then 1 else N*FACT(N - 1) . 



FIB(N) = if N = 0 then 0; 

if N = 1 then 1; 

else FIB(N - 1) + FIB(N - 2). 



. Recursive definitions of the factorial function and 
a function to compute the nth Fibonacci number. 
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process prevents it from taking advantage of local opportunities, 
or, put another way, leaves it vulnerable to unforeseen troubles like 
the combinatorial explosion just presented. 

Currently almost no work is aimed at providing language processors 

with the ability to analyze the programs they are given so that they 

might, for example, calculate Fibonacci numbers from the recursive 

* 

definition without falling into the trap just presented. 

There are two primary reasons for the lack of nonuniform 
processors. First, uniform processes are easier to understand and 
build. Being understandable, they seem trustworthy, if rigid. Also, 
where they are sufficient, they are far cheaper in computer power and 
run much faster. Second, current languages actually leave little 
decision making to the processor, so that uniform processes are 
feasible. (This may constitute one of the main differences between 
natural and artificial languages.) For instance, the Fibonacci 
example above, while valid, may seem a little strained. In fact, 
the world of computing does not abound with examples in which a 
uniform processor runs wild on a simple program. 

A flexible, competent information system that communicates with 
its users in English will necessarily be a nonuniform processor. 

The rigidity of most current natural language systems results from 
the fact that they are unifom processes in which many fundamental 

ie 

Code optimizing compilers do, in a sense, provide some of 
this ability, but they are themselves uniform processes, and treat 
simple and complex programs identically. 
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decisions are made a priori, in the design stage. If the question 
answering processes could be moved into the system’s model, and 
made the object of conversation with the user, the rigidity could 
be reduced. 

Nondeterministic Programming Languages 

It appears that the development of nonuniform processors will 
derive from work on problem solving. A problem solver is a non- 
uniform process because, in order to solve nontrivial problems, it 
must select its actions carefully on the basis of an analysis of 
its current situation and goals. Of course trivial problems can be 
solved by simply enumerating possible solutions; but in nontrivial 
problems far too many possibilities exist. Problems are actually 
stated as programs in simple nondeterministic programming languages 
by Fikes (1970) and Gibbons (1972). These programs are in turn 
executed by nonuniform processes. 

Nondeterministic languages offer increased expressiveness over 
conventional languages, because the programmer can leave substantial 
decisions to the processor. The processors, in turn, must either 
make rational decisions for themselves, rely on advice from the 
programmer or user, or else fall back on an enumeration of 
possibilities. 

The predicate calculus can be used as a nondeterministic 
programming language. Green (1968) explored the possibility of 
using the predicate calculus to represent problems, and applying 
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theorem proving techniques to find solutions. This approach is 
limited by representational difficulties due to the frame problem, 
and by the noted liabilities of uniform proof procedures. 

A very interesting direction in language development, including 

elements of the predicate calculus, LISP, and the use of advice 

from the programmer, is represented by PLANNER. PLANNER (Hewitt, 

1969) is a nondeterministic programming language that has actually 

been used to implement major systems such as Winograd’s system 

(see Winograd, 1971, for a description of this system and also for 

a good description of PLANNER) . PLANNER results form the addition 

* 

of nondeterministic elements to LISP. 

A PLANNER program is written as a set of function definitions, 
as in LISP. A PLANNER function is normally considered to be a truth 
function, so that the evaluation of a function results in TRUE or 
FALSE, with side effects. For example, to solve the problem "find 
X and y in (1, 10) such that x + y = 15," one would write a 
PLANNER function that represents the predicate calculus expression 
(Ex)(Ey)((x + y = 15) and (x e (1, 10)) and (y c (1, 10))). 
Evaluation of the function would cause the PLANNER system to prove 
that the expression is true. 

The PLANNER interpreter uses advice given in the definitions by 
the user, but is still a uniform process. The basic mechanism in 



Actually, it is Micro -PLANNER, a preliminary version of 
PLANNER, that is implemented in LISP. A full PLANNER system is 
not yet available. 
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the PLANNER interpreter is a generate and test algorithm. Fundamentally, 
then, without the addition of further mechanisms to control the search, 
the interpreter would suffer the practical deficiencies of the generate 
and test algorithm. The major method of controlling the search is the 
use of advice supplied by the programmer. This advice is attached to 
the definitions of functions, and so is scattered throughout the PLANNER 
program. The user familiar with PLANNER’S generate and test procedure 
can obtain complete control of the system’s search by adding appropriate 
advice. In addition, this advice can be manipulated by the running 
program, which suggests interesting possibilities for experimentation 
with learning. 

PLANNER, or some other suitable nondeterministic language, would 
be a good choice of language in which to write question answering 
processes to be stored in a system’s model. The processes in the 
model could be quite nonuniform, with the only uniform process being 
the programming language processor. (There is always a uniform 
process somewhere, just as behind any search process, no matter how 
sophisticated, there is always a generate and test process.) 

Conventions and Types of Data 

The question answering capability of a system derives from the 
capacity of its data structure and the processes that manipulate the 
data to obtain answers. Mediating between these is a system of 
conventions in terms of which the data are encoded and the processes 
written. 
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Conventions , Each process must know how to interpret its inputs; 



thus in finding the answer to a question, a program must know the 
significance of the data it finds. This is accomplished by specifying 
the program partly in terms of conventions about the data. For 
example, LISP*s EVALQUOTE expects arguments FN and X, where FN 
is the name of an elementary function or a lambda expression that 
defines a function, and X is a list of expressions whose values are 
to be the arguments of FN. 

The documentation of question answering systems always contains 
explanations of the system's conventions for representing information 
in the data base. For example, one finds discussions that explain, 
say, that "relation R holds between A and B" (i.e., R(A,B)) 

is represented as: A B. This is essentially a translation rule: 

it explains how to translate an assertion in English (or some query 
language) into an element or portion of a data base. 

Two observations are relevant here. First, in isolation, both 
the source and target representations are equally meaningless. They 
become meaningful only in a context that contains conventions and 
processes for interpreting them. Second, conceptually we might as 
well consider the target representation to be an assertion, so that 
the formal distinction between a datum and a (def initionally stated) 
process may be reduced or eliminated. 

Types of Data . The data base organization of question answering 
systems always contain elements that are identified as atomic objects 
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and connections between these objects. This suggests graphs; in 
fact, labeled, directed graphs are always a suitable, though some- 
times unwieldy, basis for representing a system’s data structure. 

Thus, graphs provide a vehicle for comparison of diverse data 
structures. 

On a less atomic level of discussion, the data used by question 
answering systems can usually be found in some combination of the 
following categories: atomic aggregates, formulas, processes, and 

query language syntax. 

Atomic Aggregates . Atomic aggregates are data structures 
consisting of atoms and relations between atoms, where the atoms 
and relations are constant. These aggregates are usually identified 
with either facts or object-descriptions. Some network representations 
appear to combine properties of facts and object-descriptions. 

Facts are simple elementary assertions, such as (HUMAN SOCRATES) 
of (POPULATION PODUNK 120). The defining characteristic of facts 
as opposed to described objects is that facts are stored explicitly, 
whereas object descriptions explicitly store objects and their 
descriptions, leaving various facts implicit. Thus, if we wish to 
represent the assertions that Socrates is a human philosopher, the 
fact representation might be: 

(HUMAN SOCRATES) 

(OCCUPATION SOCRATES PHILOSOPHER) 

Alternatively, we could let a description list stand for Socrates: 
((NAME SOCRATES) (OCCUPATION PHILOSOPHER) (HUMAN) ) 
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In addition to specifying what is stored, it is also necessary 
to specify how it is stored. Facts can be naturally stored as sets 
or files of n-tuples. For example, a data base containing information 
on people might include their occupations; this information could be 
stored in a file, thus: 



OCCUPATION 

JONES PROGRAMMER 

SMITH WRITER 

• • 

• • 

SOCRATES PHILOSOPHER 
0 • 

• • 

Described objects could be stored individually, with each being 
part of a lexical entry. Or, for example, lists (representing sets) 
could be stored in the data base to facilitate searching sets of 
objects . 

Network structures provide the capacity to establish essentially 
arbitrary connectivity properties in the data base. For example, in 
the ring structures of DEACON (Craig, et. al., 1966) and REL, lists 
of objects, N-tuples, and sets of N-tuples all exist in an inter- 
connected structure. 

The distinction between the types of atomic aggregates (facts, 
described objects, and networks) is practical, rather than fundamental. 
Each data base of atomic aggregates can be unambiguously represented 
by a set of atomic assertions. The varieties of structure have the 
practical purpose of establishing direct connections between related 
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assertions, and thereby eliminating the need for searching the 
whole data base for related assertions. 

Formulae . In this context a formula is an expression involving 
a variable and possibly logical connectives. ”A11 men are mortal” 
is an example of an assertion that would be represented as a formula. 
Getting information from formulae normally requires the use of some 
deductive or at least variable binding process. In a static data 
base, some formulae could be converted to atomic aggregates by 
enumerating the instantiations, e.g., tagging every item representing 
a man with mortal. However, not all the information in a set of 
formulae can be obtained in this way, and the volume of new data 
entries would swamp the system. Thus, formulae can be much more 
concise and expressive than atomic aggregates, but they require the 
availability of far more elaborate deductive processes. 

The deductive processes available today are generally uniform, 
rendering them impractical as general question answering or problem 
solving methods. Also the frame problem limits current attempts to 
represent realistic problem situations with formulas. Cordell 
Green’s thesis (Green, 1969) thoroughly explores these issues, and 
essentially carries the application of formal (predicate calculus) 
representation and uniform proof procedures to problem solving and 
question answering to their limits. 

Processes as data . Representing processes in the data base 
offers the possibility of bridging the gap between problem solving 



- 21 - 



or question answering processes and information in the data base. 
Including formulae and suitable deductive processes as data is a 
step in this direction, but the inefficiency of uniform processes 
poses a serious difficulty. With processes represented as data, 
it becomes possible to discuss, construct, and revise the represented 
processes. The uniform process (which is not in the data base) then 
can function as an interpreter for the processes in the data base, 
rather than as a problem solving process itself. This is potentially 
a major source of power for question answering and problem solving 
systems. With processes available as objects of discussion, it 
should be possible for the system to engage in verbal reasoning and 
advice-taking tasks. 

Winograd’s work is a step toward exploiting these possibilities. 
His data base is composed of facts and PLANNER theorems. A PLANNER 
theorem is an assertion, but it also contains advice to the inter- 
preter (uniform process) for evaluating (proving) the assertion. 
Winograd uses the facilities of PLANNER to obtain reasonably efficient 
searches in the solution of problems. His system can also discuss 
its actions, within limits. The addition of advice taking would be 
a major improvement to the system and would constitute a most 
interesting development in natural language processing. 

Query language syntax . The REL system uses the syntax of the 
query language to store generic and definitional information. This 
method has some very interesting properties. 
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REL uses an exhaustive bottom up parser. One can add rules to 
the grammer via the query language. Thus, the generic information 
and definitions are stored essentially as paraphrase. For example, 
if one defined the factorial function: 



def: N! = if N > 0 then N*((N - 1)!) else 1. 

then a rule would be entered that would replace ”i!" by 
"i*((i - 1)!)** if i > 0 and ”1" otherwise. Thus, in response 
to "3!" the parser would generate: 

3! 

3*((3 - 1)!) 

3*( 2! ) 

3*(2*((2 - 1)!)) 

3*(2 * (D) 

3*2 

6 



Generic information could be stored in a similar way, e.g.: 

Df : city = smoggy place. However, little actual inference can be 

accomplished because each rule that can be applied is applied. Thus, 
if we have defined the following: 

man = adult male human being 
being = creature or entity 
creature = animate object 
adult = fully developed, mature 

then the sentence "John is a man" would parse into "John is a fully 
developed, mature male human animate object or entity." 
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REL achieves some interesting, even spectacular, results using 
the grammar to allow extensions of the query language; clearly, 
though, even with the REL’s substantial control of the grammar, it 
is best to store facts in the data base and reserve manipulation 
of the grammar for the extension of the query language. 
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COMPARISON OF THREE DATA REPRESENTATIONS 



In this section three data structures present in contemporary 
question answering systems are examined, representing the following 
three alternatives: 1) data elements represent objects; 2) data 

elements represent facts; and 3) a combination of the two. Each 
of the three will be used to represent a sample data base which, 
though small, contains complexities representative of those found in 
realistic data bases. The sample data is the information in the 
following assertions: 

1) John is a person. 

2) The height of John is 6 \ 

3) John's siblings are A and B. 

4) A person has two hands. 

5) A finger is part of a person. 

The information in these assertions is somewhat ambiguous; for 
example, in Number 3 it is not clear whether John may or may not 
have other siblings. In addition, to cope adequately with these 
assertions, it is necessary to have additional information such as 
the fact that the sibling relation is symmetric. 

A data management system may either represent the data in a way 
that leaves the ambiguities intact, or use some method to resolve 
the ambiguities. Typically, the ambiguities are resolved. This can 
be done by: 1) asking the user, 2) using a built-in criterion 
or deduction to decide, or 3) failing to recognize the ambiguity 
at all, and simply storing one unambiguous interpretation. 
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The SIR Model 



In the SIR system, (Raphael, 1964), words name objects and 
relations. Objects and relations are associated by relations. 

These associations are represented by description lists in which 
each attribute names a relation and the value (s) are other objects 
or structures that enter into the named relation with the object. 
Three types of linkage between objects are allowed: 

Type 1 - The attribute has a unique value 

Type 2 - The attribute can have multiple values 

Type 3 - The attribute can have multiple described values 

Notice that Types 1 and 2 are special cases of Type 3, except 
that with Type 1 there is an implicit assertion that the attribute 
cannot have more than one value. 

In our sample data, heights would be a Type 1 relation, is a 
and sibling would be Type 2, and subpart (from ”has”) would be 
Type 3. Thus, the sample data would be represented: 

(JOHN (IS A PERSON) (HEIGHT 6') (SIBLINGS (A B))) 

(PERSON (SUBPART ((PLIST NAME HAND NUMBER 2) (PLIST NAME FINGER)) )) 

Reverse Links . The linkages represented so far are all one-way. 
To follow inverse relations (’*what persons are there?”) would 

require a search of the whole data base, unless reverse links are 

included. Thus, the data representation becomes: 
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(JOHN (IS A (PERSON)) (HEIGHT 6*) (SIBLINGS (A B))) 

(PERSON (SUBPART ((PLIST NAME HAND NUMBER 2) 

(PLIST NAME FINGER))) 

(IS A (JOHN)) ) 

(A (sibling"^ (JOHN)) ) 

(B (sibling"^ (JOHN)) ) 

This includes IS A~^ and SIBLING"^, but not HEIGHT"^. Clearly 
there is a trade-off between storage cost and search cost, so that 
the selection of reverse links to include should be made carefully. 
Perhaps it could be done automatically on the basis of frequency 
of use. 

Graph Representation . Now that a specific representation of 
the sample data has been determined, we can translate it into a 
directed graph, shown in Figure 3. 

The SIR fact retrieval and deduction methods . SIR was intended 
to make reasonable conversation. To do this, it needed the ability 
to draw reasonable conclusions from its data, as well as to answer 
specific questions. Raphael intended to select an internal 

representation that *’ had connectivity and accessability properties ** 

suitable for such processes. Further, he argues that **...the search 
programs (must be) designed to "know** the properties of the relations 
being searched, e.g., transitivity. Therefore a special set of programs 
had to be written for each relation.** 
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PERSON 




Figure 3. Graph of sample data represented by SIR. 



Raphael’s argument just quoted is not conclusive. Certainly 
the easiest way to implement the required deductive processes is 
by means of special purpose programs that are designed for particular 
relations. However, it is also possible to construct descriptions 
of relations, and apply a general deductive process that uses such 
descriptions. If such descriptions could be expressed in the query 
language, they could be included in the data base. The general 
deductive method would function as an interpreter, and the descriptions 
of relations would function as particular deductive methods. Such 
a capability should be within reach of current technology. 



Relations . Because each relation requires a separate set of 



programs, SIR was given a limited set of relations. They are: 



Set membership 

Part-whole 

Number 

Set inclusion 

Left-right 

Ownership 



(’John is a person’) 

(’A finger is part of a person’) 

(’A person has two hands’) 

(’A man is a person’) 

(’The lamp is to the left of the table’) 
(’John owns a car’) 



The inverses of these relations are also included. 

SIR can do several interesting things, such as handle exceptions 
to generic information in a reasonable way. The search program always 
looks for information about an object on the object’s description 
list. If the information is not there, the program looks at the 
descriptions of sets to which the object belongs. See Figure 4 for 
an example. 

SIR would require additional programs for the height relations 
before it could answer the question ”What is John’s height?" 
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Figure 4 



(***. there are 5 FINGERS ON EVERY HAND) 

(I UNDERSTAND) 

(***. THERE ARE TWO HANDS ON A PERSON) 

(I UNDERSTAND) 

(***. A BOY IS A PERSON) 

(I UNDERSTAND) 

(***. tom is a boy) 

(I UNDERSTAND) 

(***. DICK IS A BOY) 

(***. HARRY IS A BOY) 

(I UNDERSTAND) 

(***. tom has nine fingers) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(I UNDERSTAND) 

(***. DICK HAS ONE HAND) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(I UNDERSTAND) 

(***. HOW MANY FINGERS DOES TOM HAVE Q) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(THE ANSWER IS 9) 

(***. HOW MANY FINGERS DOES DICK HAVE Q) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(THE ANSWER IS 5) 

(***. HOW MANY FINGERS DOES HARRY HAVE Q) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(THE ANSWER IS 10) 

(***. HOW MANY FINGERS DOES JOE HAVE Q) 

(THE ABOVE SENTENCE IS AMBIGUOUS ** BUT I ASSUME (HAS) MEANS (HAS 
AS PARTS)) 

(I DON’T KNOW WHETHER FINGER IS PART OF JOE) 



Figure 4. Example of the exception principle in SIR. 
(From Raphael, in Minsky, 1968) 
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The CONVERSE Model 



The CONVERSE data base is composed of objects and relations. 

The data structure is patterned after the logical notion of a finite 
model. The set of objects is the universe of discourse, say U. If 

is an n-place relation, then in a finite model R^ c U^. CONVERSE 
would store R^ as the set of n- tuples X of objects such that 
R^ (X) is true . 

A data base containing our sample data would thus appear as: 

height = { < JOHN, 6’ > ... } 

sibling = { < JOHN, A >< JOHN, B > ... } 

person = { < JOHN > , ... } 

(Here we have taken person as a one-place predicate. We could 
also use set membership as the predicate referred to in ”John is a 
person**, and store e = {< John, person > ...} . There is no apparent 
reason to do this, however.) 

When we come to the SUBPART relation, in order to store **a person 

has two hands** and **a finger is part of a person,** we would have to 

use at least three place relation: 

SUBPART = {< person, hands, 2 >< person, finger, ?> ...} 

Here **?** is used as a place holder, to signify that the number 

of fingers per person is not known. In fact, this is still not quite 

legitimate in the absence of special search functions, because what 
is really meant is that anything that is a person must have two hands. 
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HEIGHT 6' 

• y® 



SIBLING 







JOHN PERSON 




Figure 5. 



A representation of the sample data in terms of 
CONVERSE relations. 
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This assertion could be rendered in the predicate calculus as 
Vx (person (X) ^ Subpart (X hands 2)), The interpretation of this 
formula in a finite model would be that < x, hands, 2 > would appear 
as an element of the relation SUBPART for each x such that <x> 
appears in person. 

A reasonable question answering system using this data base 
organization would use special search functions to avoid the above 
difficulties. This being the case, the data structure is no longer 
strictly a finite model. 

It is not clear what advantages are to be gained from the finite 
model structure. The logician's purpose in excluding relations 
from the universe U is to assure that the finite model is 
unambiguous, and the truth of an assertion can be calculated directly, 
without the need for deduction. Properties of predicates such as 
symmetry of two place predicates are therefore properties of models, 
rather than elements of models. The question answering system 
builder's purpose is different: he wants to be able to store and 

recover facts easily and make at least simple deductions easily. 

Since the objectives of the logician are so different from those 
of the system builder, it would be surprising if a strict finite 
model were particularly appropriate for question answering. On the 
contrary, there are some significant disadvantages. For instance, 
in a finite model, relations are not objects. Thus, the data 



Note that in the finite model, PERSON = {< person^>, < person 2 >. 
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cannot assert directly that a given relation has a certain property - 
e.g., "sibling is symmetric" or "a person has two hands." 

Another disadvantage with CONVERSE relations is that, once 
specified, they are fixed forever. In ordinary conversation, we feel 
free to use as many or few modifiers and modalities in our assertions 
as suits our purpose. If CONVERSE specified a place in each relation 
for each possible argument, the n-tuples would be long and often full of 
place holders. Leaving such places out of the relation prevents 
CONVERSE from using the modality at all. Thus if SUBPART were a two 
place relation, then the fact that a person has two hands could not 
be represented. The SUBPART relation would appear as: 

SUBPART = {<person, handxperson, finger> ...} 

Other systems use data structures that allow the flexibilities 
mentioned here. Compare, especially, Winograd (1971) or Quillian 
(1966). 

One of the principal advantages claimed for this data structure 
in the CONVERSE literature is that rapid lookup can be achieved by 
means of binary search. However, this advantage results from storing 
long, ordered arrays of similar elements, not from the fact that the 
elements are n-tuples. This advantage could be secured as well if 
the elements stored were small networks rather than n-tuples. This 
would allow for direct storage of modalities: 

SUBPART = {<person, f inger> , <person, hand, (mode (number 2))> ...} 
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Such a scheme appears to capture the claimed advantages of the 
finite model, while serving the question answering goals more directly 
than a pure finite model can. 

Rings 

The ring structures considered here were used in the second 
DEACON system (Craig et. al. , 1966), and in the first REL system. 

The ring structure was developed from the original list-structured 
data base used in the first DEACON system. The motivations for 
developing the ring structures were primarily these: (1) over- 

specialization in the list-structured data base was (wrongly, I 
believe) blamed on inherent limitations of list structures such as 
those available in IPL/V and LISP. Ring structures were intended 
to overcome these supposed limitations. (2) Ring structures provide 
much more connectedness in the data; this allows the retrieval functions 
to follow more direct paths within the data. 

The rings are implemented as two-dimensional circular lists, i.e., 
each list word contains a symbol and two links, and the lists are 
circular in both dimensions. Because of the double linking, the rings 
cannot be conveniently represented in LISP notation. The simplest 
direct notation is a graph. The sample data represented in rings is 
shown in Figure 6. 

The solid rings represent objects, sets, and properties (JOHN, 
PERSON, and HEIGHT, respectively). The dotted rings represent facts, 
such as (JOHN SIBLING A) , (JOHN PERSON) . Each dotted arrow carries 
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Figure 6. The sample data represented in rings. 
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the name of the ring to which it points - the arrow itself points 
to a particular point within the ring. 

Note that there are only two types of dotted rings - those of 
length two and those of length three. The conventional interpretations 
are (<ob j ect><attribute><value>) and (<object><superset>) . 
Consequently, the data base represents the facts in the sample data 
quite ambigously. Thus, "John is a person" and "person is a John" 
result in the same data. Such nonsense had to be eliminated at the 
syntatic level in DEACON and REL. 

The basic ring structure represents only elementary facts, and 
in such a way as to require S3mtactic help to provide unambiguous 
interpretation. Consequently, very little a-priori information is 
contained in the data structure itself. Thus, the relations (such 
as HEIGHT) are themselves data elements, rather than being parts of 
the data structure. Another consequence is that no inference 
capability is included because such capability would require pre- 
established conventions for interpreting the data. Thus, the tran- 
sitivity of LOCATION, if it is to be available, must be supplied 
outside of the data base, and generic information such as "a person 
has two hands" does not fit into the data base at all. 

Generic Information . Within the basic ring structure there is 
no explicit allowance for generic information. Thus, while one could 
add a fact (PERSON SUBPART FINGER) to represent "a person has a finger", 
the data would not show the distinction between "each object that is 
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a person has a finger" and "the object person has finger". In SIR 



this type of distinction was handled by a search routine associated 
with the SUBPART relation; no such individual search functions were 
used in DEACON or REL. 

Time-dependent Information , The retrieval and linguistic methods 

of DEACON and REL allowed them to cope with assertions about facts 

that changed with time. To deal with such facts, the ring structure 

was extended to allow the ( <object><attribute><value>) ring to 

contain a time modifier. The result was that the assertions "the 

location of A is X from t to t^" and "the location of A 

o 1 

is Y after would be represented as in Figure 7. 

These systems contained built-in functions to search such structures. 
The tense information in a sentence was encoded in terms of markers 
such as BEGIN and END. Some modest inference ability was included to 
allow the system to answer "yes" on the basis of the above data if the 
current time t is greater than and the question is "Is A at 

Y?" Neither system provided any approach to the frame problem. The 
frame problem was never troublesome because the systems made 
essentially no deductions and the data on which they worked contained 
no temporal inconsistencies. 

Modalities . The assertion "a person has two hands" presents the 
same difficulties in rings that it did in the CONVERSE data structure. 

The fact asserted could be taken as a three place relation, i.e.. 
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Figure 7. Example of time-dependant information in rings. 



(SUBPART PERSON HAND 2); however, such convention would restrict the 
last place in the relation to a number. A more flexible approach is 
needed. An extension of ring structures similar to that used to 
represent time-dependent information could be used to allow modalities, 
and still avoid the difficulties that arose in the CONVERSE repre- 
sentation. Figures 8 and 9 show two possibilities. 

In Figures 8 and 9 the two assertions **a finger is part of a 
person** and **a person has two hands** are represented. In Figure 8, 
the two facts are combined in a single complex value for the attribute 
SUBPART using the same structure that was used to represent time- 
dependent facts. As this example indicates, the representation of 
time-dependent facts can be used to contain other types of modal 
information by extending the (<object><attribute><value>) type ring 
to be (<object><attribute> ( (<mode><value>) . . . (<mode><value>) ) ) . 

Clearly, also, the need for the ability to make simple deductions 
increases with the elaboration of model information. In Figure 9, 
simple and complex values are mixed in a different but also plausible 
way, which illustrates the need for conventions for interpreting a 
data structure. This need is more stringent for more complicated 
structures . 

The ring structure could support other types of elaborations, 
such as the part-whole assertions in our sample data. However, such 
elaborations would require corresponding special storage conventions 
and search functions. 
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Figure 8. 



Possible 



representation of modalities in rings. 
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I* FINGER 

I person' 




Figure 9. Possible representation of modalities in rings. 
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SUMMARY 



The long range objectives of question answering research are 
difficult to spell out with precision, because later developments 
will naturally have considerable influence on the directions taken 
by the technology. At this time it appears that the lack of 
expressiveness and flexibility of the data base structures constitutes 
a primary restriction on the performance of question answering systems. 
Compared to the data structure problems, the problems of syntax and 
parsing are well understood. 

With the development of better data structures will come more 
powerful and competent question answering processes. Question 
answering will merge with problem solving. The long range goal is 
the development of systems that will serve us as intelligent consultants. 

Current Technology 

Current question answering systems are able to apply various 
counting and sorting operations to data that could be simply 
represented in the form of charts, lists, and tables. In the best 
systems the linguistic capabilities are more than adequate for this 
task. However, deductive capabilities are almost totally absent, as 
are the abilities to solve problems and make generalizations from the 
data. Question answering processes are designed into the basic 
structure of the systems, and the systems are unable to discuss these 
processes with the user or to change them to suit the user's con- 
venience. These systems generally give users the impression that 
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they are very rigid and unforgiving, use language in strange ways, 
and are difficult to use. Users get the impression that they must 
conform to the system, rather than the reverse. 

Research Problems 

The problem of scale . The systems that provide the most 
interesting capabilities generally make large demands for computer 
power, in both time and memory. Consequently it has not been 
possible to bring the best question answering capabilities available 
to bear in trial applications of realistic size. It is not clear how 
expensive of computer power intelligent processes need to be; perhaps 
they are intrinsically expensive. In any case, it is necessary to 
investigate ways to apply current question answering capabilities to 
large data bases. This problem will probably never be solved; there 
will probably always be applications that are just too large to be 
accommodated by available systems. 

Data Structures . The examination of example data structures 
earlier in this paper showed that the diversity and complexity of 
information representable in those structures is limited. This 
limitation is a critical obstacle to the development of competent 
question answering systems. The addition of processes as data 
objects would extend the diversity of data, and also perhaps open 
the way to putting the question answering processes themselves in 
to the data base and making them available for discussion and 
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modification by the user. This direction of research would also 
ease the eventual introduction of true problem solving processes 
into question answering systems. 

Applications . Question answering research could be stimulated 
and would receive much valuable direction if there were more systems 
in actual use. In addition, potential users who do not now have 
question answering systems at their disposal could benefit from 
increased capabilities such systems could offer them. But the 
potential user who’s operation could be greatly enhanced by a 
question answering system and the scientist who could provide a 
suitable system often know nothing of each other’s problems and 
capabilities. An effective way to bridge the gap would be to form 
teams of people selected from both groups. Such a team could scout 
for possible applications, evaluate the potential benefit, and 
provide a pilot system in a short time, if appropriate. 
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